feat(compositor): port the native compositor to Linux (Vulkan backend) - #183
Merged
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
EtienneLescot
force-pushed
the
feat/linux-compositor-port
branch
from
July 28, 2026 07:49
19c8314 to
f3aa8e5
Compare
EtienneLescot
changed the base branch from
release/v1.8.0
to
feat/d3d-warp-fallback
July 28, 2026 07:49
EtienneLescot
added a commit
that referenced
this pull request
Jul 28, 2026
Closes part of PR #183 (the scoping body). Defines the target architecture (wgpu backend mirroring the D3D11 seam #162 introduced), the per-platform frame contract, the WGSL shader port inventory (1:1 with shaders.hlsl's 9 entry points), the cosmic-text replacement for text.rs, the build/CI plumbing for Linux, and a WP0..WP7 milestone plan. Adds a Linux port subsection to native-compositor.md and a doc-table entry to README.md. The spec body of PR #183 is updated separately to reference this doc.
EtienneLescot
added a commit
that referenced
this pull request
Jul 28, 2026
…d skeleton Implements the foundation of PR #183. WP0 — Cargo target gating - workspace deps: wgpu 24 (cross-platform, Vulkan/Metal/D3D12; pinned to the windows-0.58-compatible line so the existing D3D11 path stays on its validated toolchain), pollster for block_on. - compositor/Cargo.toml: wgpu + pollster deps cross-platform, the windows dep kept for the D3D11 path. - compositor-view-napi: the windows dep stays; lib.rs is split — the existing napi surface is cfg(windows)-gated into windows.rs, a non-Windows placeholder stands in until WP7 ships the Linux napi skeleton. - compositor/src/lib.rs: cfg(windows) on d3d/cpu_frames/text/compositor/ live/pipeline (the D3D11 path); portable modules + new vk* stay ungated so the crate compiles everywhere. - poc-d3d: cfg(windows) on the Win32 body; non-Windows stub so `cargo check` on the target stays green. - compositor/build.rs: cross-build detection via cfg!(host_os) vs CARGO_CFG_TARGET_OS. When target != host, skip cc/bindgen and write a cached or stub ffi.rs so `cargo check --target x86_64-unknown-linux-gnu` from a Windows host passes without a cross-toolchain. Path separator made platform-aware. WP1 — vk.rs (wgpu device + probe + lavapipe classification) - VkGpu { device, queue, backend, adapter_info } mirrors d3d::Gpu shape. - create (hardware-strict, for tests/goldens), create_blocking, create_auto (fallback HW -> lavapipe; full semantic equivalence with d3d::Backend in WP1+). - probe cached via OnceLock; returns the same "none" semantics as a missing addon on a machine without a Vulkan implementation. - classify() detects lavapipe by adapter name on Linux (llvmpipe / lavapipe substrings) and WARP on Windows — both routed to BackendKind::Cpu so probeBackend() reports "cpu" exactly as the Windows WARP rung does today. - Unit tests: probe-doesn't-panic, classify-hardware, classify-lavapipe. WP2 — vk_frames.rs (NV12 upload skeleton) - VkFrameTex { y: r8unorm, uv: rg8unorm, width, height } mirrors the D3D11 NV12 texture that cpu_frames.rs presents today; the AVFrame carrier (data[0] = Box ptr) is the contract #162 introduced, generalised to carry the platform payload through crate boundaries. - VkFrames: AVFrame slot (av_frame_alloc), ensure_textures(w, h) creates the two R8/Rg8 textures when dimensions change. - Drop frees the AVFrame slot. - The swscale -> AV_PIX_FMT_NV12 -> queue.write_texture path lands in a follow-up WP2 commit; this commit establishes the seam and types. vk_render.rs — module skeleton with VERSION constant and a test. vk_shaders/blit.wgsl — minimal placeholder WGSL (fullscreen triangle + black clear) for the include_str! source of the slice renderer. Verified: - `cargo check --workspace` green - `cargo check --workspace --target x86_64-unknown-linux-gnu` green (uses OPENSCREEN_BINDGEN_CACHE_PATH to feed the bindgen output from a prior host build into the cross-build — see build.rs write_stub_ffi.) WP3+ (slice renderer with WGSL core shaders and cross-backend golden diff) is scoped in linux-compositor-port.md but lands in follow-up commits to keep this PR reviewable.
EtienneLescot
added a commit
that referenced
this pull request
Jul 28, 2026
…ackend skeleton The cross-backend architecture for the Linux port. The trait `CompBackend` is the seam between the orchestration math (which stays in `Compositor`) and the per-platform GPU work (which moves behind the trait). Two impls exist: - `D3d11Backend` (Windows): wraps the existing `Compositor` and forwards each trait method. The 3000-line `compositor.rs` body is preserved byte-for-byte — `upload_cb` becomes `pub(crate)` and the `LayerCB` struct moves to `backend/mod.rs` so both backends see the same type. All 86 tests still pass on Windows with no behavioral change. - `WgpuBackend` (Linux/macOS, cfg-gated to `not(windows)`): implements `CompBackend` using the same wgpu primitives as `vk_render.rs`. The 8 minimal trait methods (`render_size`, `bind_compose_state`, `begin`, `upload_cb`, `fs_pass`, `blur_bg`, `draw_video`, `draw_solid`, `draw_shadow`, `draw_quad_shadow`) cover the compose_frame path. The slice test continues to use `vk_render::render_slice_c1` directly for now (proven working); the trait-dispatched path will be wired up in WP4+ as `compose_frame` migrates. ### Trait scope (PR #183, première vague) The trait covers **the compose_frame path** — the 8 methods that `compose_frame` invokes to assemble a frame. Out of scope (still on `Compositor` directly, will migrate as needed): - encode-side (`fs_pass` for `ps_y`/`ps_uv`/`ps_blur`) → WP6 - background blur chain (`blur_bg` Kawase) → WP4 - cursor draws (sprite, themed, math) → WP4 - image background (`draw_image_bg`) → WP4 - annotations (`draw_annotations`) → WP5 (text_cosmic) + WP4 This minimal scope protects the Windows path: the refactor is borné et testable. `WgpuBackend` n'expose pour l'instant que ce qu'il sait faire (les opérations du slice WP3). ### Cross-platform types `LayerCB` (the 128-byte constant buffer / uniform struct) is now defined in `backend/mod.rs` as `pub struct LayerCB` (platform-neutral — same layout as the HLSL `cbuffer Layer` and the WGSL `struct Layer`). `compositor.rs` imports it via `use crate::backend::LayerCB;` (not `pub use`, which would create a new type). `VkGpu` now derives `Clone` (cheap Arc inside `wgpu::Device`/`wgpu::Queue`) so backends can share it cheaply. Verified: - `cargo check --workspace` (Windows host) green. - `cargo check --workspace --target x86_64-unknown-linux-gnu` green (`WgpuBackend` compiles non-Windows). - `cargo test --tests -p openscreen-compositor`: 86 tests pass — 80 unit (unchanged), 3 integration (unchanged), 1 vk_cross_golden slice (unchanged), 2 vk::tests (unchanged). - The Windows `D3d11Backend` is **a forwarder only** — no D3D11 method body was copied. Behavior is bit-identical to before this commit. - The `vk_cross_golden` slice test still passes against the `vk_render` standalone renderer; it is the production path's responsibility (WP4+) to wire `compose_frame` through `CompBackend`. This is the architecture commit. WP4+ extends both backends to fill in the unimplemented methods (blur chain, cursor, image bg, annotations, encode-side) and wires `compose_frame` / `live.rs` / `pipeline.rs` / `text.rs` / `napi` to dispatch through `Compositor::b: Box<dyn CompBackend>`. The trait surface (10 methods) is enough to demonstrate the architecture end-to-end while keeping the diff reviewable. A full `compose_frame` migration would be a follow-up that lifts every method body into the backend (and is the bulk of WP4+).
EtienneLescot
added a commit
that referenced
this pull request
Jul 30, 2026
PR #183 rebased from #162 base (old branch) to release/v1.8.0 (the target the user asked for after conflict resolution on #162). The 41 extra commits between #162 and release/v1.8.0 are NOT in this PR — they represent upstream work (native GIF export pipeline, pixi.js removal, zoom/cursor fixes, etc.). The port reuses the same architecture and files (vk.rs, vk_decode.rs, vk_frames.rs, vk_render.rs, text_cosmic.rs, backend/{mod.rs,d3d11.rs,wgpu_backend.rs}, tests/vk_cross_golden.rs, live_lin.rs, pipeline_lin.rs, compositor-view-napi/src/linux.rs) but on top of the new base. Key adaptations vs the old base: - `compositor`/`d3d`/`live`/`pipeline`/`text`/`gif_export` are now `#[cfg(windows)]` modules in `compositor/src/lib.rs`. The Linux sibling modules (`vk*`, `backend`, `live_lin`, `pipeline_lin`, `text_cosmic`) are platform-agnostic (cross-compile via wgpu). - `compositor::LayerCB` moved to `backend::LayerCB` (the platform-neutral definition matches HLSL `cbuffer Layer` and WGSL `struct Layer` byte-for-byte). Compositor's `LayerCB` is now `use crate::backend::LayerCB`. - `compositor/src/compositor.rs` uses `use crate::backend::LayerCB` instead of defining its own. This unifies the two `LayerCB` symbols that previously caused a `E0308` mismatched-types error in d3d11.rs. - `pipeline` (D3D11 arm) is NOT renamed to `pipeline_win`. The D3D11 `run_composited`/`run_composited_multi` keep their original names and signatures. The Linux path (`pipeline_lin::Decoder`/`VideoEncoder`) is parallel. - `compositor-view-napi`: split into `lib.rs` (cfg-gated `mod windows` / `mod linux` + `pub use`) plus `windows.rs` (full Windows addon, 622 lines) plus `linux.rs` (slice-based Linux addon, 230 lines). This replaces the previous "stub + Windows content in lib.rs" pattern the old branch had. The Windows dependency is `target.cfg(windows)` gated, so Linux builds don't pull `windows = "0.58"`. - `compositor/build.rs` cross-build-aware: detects `cfg!(windows)` vs `CARGO_CFG_TARGET_OS` and either runs the full bindgen path (native host) or writes a cached/stub `ffi.rs` (cross-build, via `OPENSCREEN_BINDGEN_CACHE_PATH`). The wrapper.h includes `<libswscale/swscale.h>` (needed for `vk_frames.rs` swscale calls). - `compositor/wrapper.h` includes `<libswscale/swscale.h>` for swscale bindings used by `vk_frames.rs`. The build.rs allowlists `sws_.*`. - `cosmic-text = "0.19"` added as a workspace dep + compositor dep (the v0.19 features are default — there's no `fontdb`/`swash` feature flag in v0.19). - `poc-d3d`: bench.rs's `export_gif` call updated to the new signature (`&[ClipSource]` instead of `&screen, &webcam, Some(&cursor)`). `gpu`/`comp` are now hoisted early so the GIF shortcut can run before the rest of the MP4-only setup. `mod app;` / `mod bench;` in `poc-d3d/src/main.rs` and the `windows` dep in its `Cargo.toml` are now `#[cfg(windows)]`/target-gated so Linux builds skip them. What's in this branch (new files only): - `crates/compositor/src/vk.rs` — wgpu device + lavapipe probe (`BackendKind::Cpu` when adapter is software via lavapipe on Mesa). - `crates/compositor/src/vk_decode.rs` — `SwDecoder` (software ffmpeg decode via `avcodec`). - `crates/compositor/src/vk_frames.rs` — `VkFrames` (NV12 split upload to two wgpu textures, AVFrame carrier). - `crates/compositor/src/vk_render.rs` — `render_slice_c1` (slice renderer: WGSL layer.wgsl pipeline + bind groups + readback to PPM). - `crates/compositor/src/vk_shaders/{layer,blur}.wgsl` — WGSL ports of the HLSL `ps_main` mode 0/1/2 + Kawase chain. - `crates/compositor/src/text_cosmic.rs` — cosmic-text-based Linux text rasterizer (rustybuzz + swash + fontdb). - `crates/compositor/src/backend/{mod,d3d11,wgpu_backend}.rs` — `CompBackend` trait + D3d11Backend forwarder (wraps existing Compositor) + WgpuBackend Linux impl. - `crates/compositor/src/live_lin.rs` — Linux LiveView skeleton (cfg(not(windows))). - `crates/compositor/src/pipeline_lin.rs` — Linux pipeline arm (Decoder / VideoEncoder / ExportCodec / run_composited with VAAPI candidates: `h264_vaapi` + `libopenh264`). - `crates/compositor/tests/vk_cross_golden.rs` — slice integration test (decode frame 180 of the fixture, render, write PPM, log stats). - `crates/compositor-view-napi/src/{linux.rs,windows.rs}` — split napi addon arms per platform. Verified: - `cargo check --workspace` (Windows host) — green. - `cargo check --workspace --target x86_64-unknown-linux-gnu` (via `OPENSCREEN_BINDGEN_CACHE_PATH`) — green. - `cargo test --tests -p openscreen-compositor` — 106 tests pass (100 unit + 2 export_timing + 3 output_geometry_golden + 1 vk_cross_golden slice). The `vk_cross_golden` slice produces a 1920×1080 RGBA8 PPM with mean RGB ≈ 232 on this Windows AMD iGPU. What this branch deliberately does NOT include (WP4+–WP7 follow-ups per `linux-compositor-port.md` §7): - Full effect set on `WgpuBackend` (modes 4/6/7/8/9/10/11/12/13 — only modes 0/1/2 and blur are ported from HLSL today). - `compose_frame` orchestration port to trait dispatch (the 800-line pure Rust math in `compositor.rs::compose_frame` still runs through the D3D11 path). - Full Linux LiveView (multi-frame render thread) — `live_lin.rs` is a skeleton today. - Linux MP4 muxer full implementation (ffmpeg avformat_new_stream + write_header + write_trailer) — `pipeline_lin::run_composited` is a skeleton that opens the encoder + runs the 1-frame path; the full muxer lands in WP7. The architecture (`CompBackend` trait + `LayerCB` + cross-platform `vk_*` modules + `vk_cross_golden` end-to-end test) is in place to absorb each of these as small focused follow-up commits.
EtienneLescot
force-pushed
the
feat/linux-compositor-port
branch
3 times, most recently
from
July 31, 2026 20:54
d47e5e4 to
56a736d
Compare
EtienneLescot
marked this pull request as ready for review
July 31, 2026 20:57
EtienneLescot
changed the base branch from
feat/d3d-warp-fallback
to
release/v1.8.0
July 31, 2026 20:58
EtienneLescot
force-pushed
the
feat/linux-compositor-port
branch
2 times, most recently
from
July 31, 2026 21:23
d5bf7f7 to
513495d
Compare
EtienneLescot
force-pushed
the
release/v1.8.0
branch
from
July 31, 2026 21:35
157f11b to
91741a4
Compare
…nd architecture
release/v1.8.0's macOS/Metal port unified the cross-backend layer differently
from the original Linux port: per-platform modules aliased to generic names
(compositor / d3d / text / pipeline / cpu_frames), a shared
frame_geometry::plan_frame (the geometry half of compose_frame, iso-render
across backends), and shared live.rs + compositor-view-napi. This re-expresses
the Linux port on that architecture instead of the old parallel CompBackend
trait.
New per-platform Linux modules (cfg(target_os = "linux"), aliased in lib.rs):
- d3d_linux (Gpu/Backend, wgpu/Vulkan) from vk.rs
- linux_frames (CpuFrames, NV12-split upload) from vk_frames.rs
- text_linux (cosmic-text rasterizer) from text_cosmic.rs
- compositor_linux (renders FrameGeometry via WGSL) from vk_render + layer.wgsl
- pipeline_linux (Decoder; export is a WP6 stub) from pipeline_lin + vk_decode
plus linux_decode (SwDecoder helper) and vk_shaders/{layer,blur}.wgsl.
build.rs gains a Linux branch (wrapper_linux.h; ffmpeg without D3D11VA/VideoToolbox);
compositor/Cargo.toml gains cfg(linux) deps (wgpu / pollster / cosmic-text);
poc-d3d (the Win32 bench POC) is cfg(windows)-gated so the workspace builds on Linux.
Verified on Linux (dzn/AMD): `cargo check --workspace` is green, and the
compose_linux test renders a real 960x540 frame through the shared plan_frame
geometry (mean_R = 202.6). compose_frame draws the core today (solid background
+ screen cover-fit with rounded corners); webcam PiP, cursor, annotations
(WGSL mode 11 text), background blur and 3D tilt reuse the same draw primitives
and land incrementally -- now sharing the Metal port's cross-backend plumbing
instead of a parallel Linux one.
…e 11) compose_frame now draws the scene's text annotations over the screen layer, mirroring the "text" branch of compositor_macos: each annotation is placed relative to the screen rect (g.s_dst), its TextSpec is rasterised by text_linux (cosmic-text -> R8 coverage atlas), and drawn with layer.wgsl mode 11 tinted by the annotation colour. Bind groups are built before the render pass (one uniform buffer per draw). Differs from the Metal path only in the texture format: macOS bakes colour into a BGRA texture; here the atlas is R8 coverage and the shader tints it (cb.color) -- same visual result, and the iso-render contract is on the shared plan_frame geometry, not on text rasterisation. Image/figure/blur annotations and the text_anim reveal/opacity animation remain incremental.
Webcam layer (mode 0) placed at plan_frame g.w_dst/g.w_radius, gated by g.shape_fade>0, drawn over the screen. cover-crop UV, mirror, shadow and circle shape remain incremental; compose_linux test now writes a PPM for visual inspection.
Restructure compose_frame into bg-pass (clear + gradient mode 5) -> blur_bg (3 down / 3 up Kawase pyramid, blur.wgsl) -> fg-pass (screen + webcam + annotations, LoadOp::Load). Gated by cfg.bg_blur. Adds the WGSL mode 5 linear gradient to layer.wgsl. 1:1 port of the Windows/macOS blur_bg. compose_linux test now renders a gradient background + padding so the blurred background is visible around the inset screen.
Draw the cursor as a themed RGBA sprite on top of the composite: plan_cursor -> upright placement -> load_image_texture (PNG/data-URI via the image crate, cached in img_cache) -> LayerCB mode 7. Adds WGSL mode 7 (sprite sampled on texY, Clip-to-canvas via fx). 1:1 port of the macOS draw_cur_themed upright path; tilted (mode 13) and motion-blur (taps>1) are follow-up increments. compose_linux test now also renders a green data-URI sprite at screen center and asserts the green pixels appear.
Implement run_composited_multi for Linux: software VideoEncoder (libopenh264 / libkvazaar, the LGPL-build encoders that need no HW device), the composited RGBA frame read back (readback_direct) and converted to YUV420P via sws_scale, fed through the SHARED walk_composited_timeline into an MP4 muxer (avformat + the sn_fmt_set_pb C shim, as on Windows/macOS). Video-only for now; AAC audio mux is the next increment (audio.rs / AacEncoder are already shared), as is VAAPI/Vulkan hardware encode. Verified: export_linux_mp4 test renders ~1s of the fixture -> a valid 30-frame H264 MP4 (ffprobe: h264 640x360, 30 packets), content correct.
Wire the shared audio machinery (audio.rs: AacEncoder, decode_clip_audio, stretch_clip_pcm_by_speed, build_audio_concat_plan, assemble_concatenated_pcm) into run_composited_multi: add the AAC stream before the header, collect one PCM per clip in on_clip_end (speed-stretched to the frames the clip actually produced), then a single AAC encode after the video walk. Silent track when a clip has no audio — parity with Windows/macOS which always mux AAC. Verified: export_linux_mp4 (has_audio=true) yields an MP4 with h264 (30 pkts) + aac (48 pkts), duration 1.0s.
…tor_linux Handle SceneBackground::Image in compose_frame: load the wallpaper via load_image_texture (reused from the cursor), cover-fit its UV rect to the output aspect, draw it as WGSL mode 6 (opaque RGBA on texY) in the bg-pass. The bg handling is refactored into a BgLayer/BgDraw pair covering gradient (mode 5) and image (mode 6) uniformly. 1:1 port of the macOS draw_image_bg cover math. Verified: compose_linux_fond_image renders an orange data-URI wallpaper filling the background around the inset screen (orange pixels 487 -> 152970 once the scene padding is wired through set_live_params).
`Math.round` returns negative zero for any delta in [-0.5, 0), which is routine under fractional scaling where screenY deltas are fractional. V8's IsInt32() rejects -0, so gin refuses the conversion and BrowserWindow's native setPosition throws "Error processing argument at index 1, conversion failure from" — fatal, because installMainProcessErrorGuards re-throws. `Number.isFinite(-0)` is true, so the existing finiteness guard could not catch it. `| 0` collapses -0 to 0 and pins the value to int32. Reproduced on Electron 41.2.1: flipping which axis carries the -0 flips the reported argument index, and the empty tail after "from " identifies the bad value as a Number rather than undefined/null.
The HUD root carried `-webkit-app-region: drag` while the grab handle explicitly opted out with `no-drag` — exactly inverted. One cause, both reported symptoms on Wayland: - pressing empty space beside the bar dragged the HUD, because the root is the whole 820x560 window and a compositor honours a drag region whether or not anything is painted there. It reads as "dragging the window behind". - pressing the grab handle did nothing, because it fell through to the IPC/setPosition path, which Wayland prohibits: getPosition() answers [0,0] and setPosition() only updates Electron's own cache. Windows and macOS never saw this: setIgnoreMouseEvents makes the transparent area input-transparent at the OS level there, so the root region was never pressable. That call is a no-op on Wayland. The handle now takes the native drag region only where the platform cannot position itself; Windows and macOS keep the pointer-handler path unchanged, since a drag region swallows pointer events and the two cannot coexist.
`crates/thirdparty/` holds the LGPL ffmpeg build the Rust compositor links against (see crates/.cargo/config.toml). It is gitignored but still watched: several thousand files, including a doc/ directory full of HTML. Extracting it fires the same reload storm the neighbouring worktree entries already guard against, leaving the running app stale.
blur.wgsl, layer.wgsl and linux_decode.rs were written from Windows and carried CRLF plus double-encoded French comments: an em dash (U+2014) had been decoded as Mac Roman and re-encoded as UTF-8, so "—" read as "ÔÇö" and the accented characters were mangled with it. .editorconfig already mandates `end_of_line = lf` and `charset = utf-8`, but nothing enforced it on checkout. .gitattributes now does, so this cannot silently come back the next time someone builds on Windows. Comment text and line endings only — no code or shader logic changes.
blur.wgsl's vs_main read `pos[vid]` twice. naga 24 keys its `spilled_composites` map by the base expression handle, so the second dynamic access overwrites the first entry: a single OpVariable is emitted while the first access's OpStore/OpAccessChain still reference an id that has no definition anywhere in the module. The module therefore violates VUID-VkShaderModuleCreateInfo-pCode-08737 and driver behaviour is undefined. RADV dereferences the dangling id and segfaults inside vkCreateGraphicsPipelines, killing the process before the compositor finishes construction; lavapipe happens to survive, which is why the crash looked adapter-specific. No Mesa version fixes this and no Mesa bug exists for it — the SPIR-V we hand every Vulkan driver is simply malformed. Doing the index once and reusing the value is pure common-subexpression elimination, so the output is pixel-identical. layer.wgsl was never affected: its vs_main derives the quad corners arithmetically and has no array literal, which is why only the blur pipeline ever crashed. Fixed upstream by gfx-rs/wgpu#7239 (wgpu 25). There is no naga 24.0.x patch release, so wgpu 24 can never pick it up and the shader must avoid the construct until that upgrade lands. Verified on AMD Radeon 610M (RADV RAPHAEL_MENDOCINO, Mesa 25.2.8): the four compose_linux tests went from dumping core to 4/4 passing on the default adapter, with no VK_DRIVER_FILES override.
The Linux counterpart of the Windows and macOS addon builds, which had no equivalent — so compositor_view.node was never produced on Linux and compositorViewService found nothing to load. The whole native compositor was unreachable from the app there, however well the Rust crate itself worked. Two things differ from Windows, and both are load-bearing: - FFMPEG_DIR is not supplied by crates/.cargo/config.toml, which pins the win64 tree only (cargo's `[env]` has no per-target form). The script resolves an explicit FFMPEG_DIR or a vendored thirdparty/ tree, and says what to do when neither exists. LIBCLANG_PATH and clang's missing stddef.h are discovered the same way rather than left to each contributor. - Shared libraries are found by RUNPATH, not PATH. Windows prepends the DLL directory to PATH at runtime (ensureFfmpegSharedDllsOnPath), but glibc reads LD_LIBRARY_PATH once at process start, so that trick cannot work once Electron is running. The addon is instead linked with `-rpath,$ORIGIN` and the five ffmpeg sonames are copied beside it, which makes the .node self-contained wherever it lands — no env var, no PATH surgery. Verified on AMD Radeon 610M (RADV, Mesa 25.2.8): the produced addon loads under Electron, exposes all twelve entry points, and probeBackend() answers "hardware" — a real GPU device, not a software fallback.
On Linux the native compositor called into `libffmpeg.so` — Chromium's own
heavily stripped ffmpeg — instead of the LGPL build shipped beside the addon.
ELF has a single flat symbol namespace and `libffmpeg.so` is a DT_NEEDED
dependency of the Electron binary, so it is loaded into the global scope
before any addon can be dlopen'd. Every `avformat_*` / `avcodec_*` / `av_*`
symbol the addon imports was already satisfied by the time its own RUNPATH
was consulted.
Confirmed with LD_DEBUG=bindings:
binding file .../compositor_view.node
to .../electron/dist/libffmpeg.so: symbol `avformat_open_input'
The visible symptom was the editor showing "Preview unavailable on this
machine" with avformat_open_input returning -1330794744, which decodes to
AVERROR_PROTOCOL_NOT_FOUND: Chromium's ffmpeg has no `file` protocol, since
Chromium feeds it buffers rather than filenames. The quieter danger was that
the addon ran against an entirely different ffmpeg than the one it was
compiled against — no libopenh264, no libkvazaar, and no guarantee the two
agree on anything beyond a shared soname. It did not fail louder only because
the major versions happen to match.
RTLD_DEEPBIND puts the addon's own dependency scope ahead of the global one,
so the `$ORIGIN` RUNPATH wins and it binds to the ffmpeg it was built with.
After the change the same trace reads:
binding file .../compositor_view.node
to .../electron/native/bin/linux-x64/libavformat.so.62
Windows and macOS are structurally immune — PE resolves imports by DLL name
and Mach-O records the defining dylib per symbol — so they keep the plain
require and are untouched.
Every Linux recording is unseekable. Capture falls back to getDisplayMedia/MediaRecorder — there is no native Linux capture helper, where Windows has wgc-capture and macOS has screencapturekit — and MediaRecorder writes WebM as a live stream, so the file carries neither Cues nor SeekHead. Verified by scanning the bytes of a real recording: both absent. `av_seek_frame` therefore fails for any non-zero timestamp, and `decode_at` turned that into a hard error. Scrubbing the preview died on it, and export failed partway through — around 85% of the file, which read as "the export crashes at the end". Rewinding to 0 stays possible without an index, and the loop below already scans forward to the target. The cost is linear, which is only acceptable because the decoder no longer re-seeks per frame: at ~0.07 ms/frame, reaching second 14 costs tens of milliseconds instead of failing outright. Verified on the real unindexed recording: a 900-tick soak with periodic seeks now runs 894 frames with no seek error and a clean teardown. This is a fallback, not the fix. Re-muxing through the Matroska muxer writes the index in ~0.06s with no re-encode and no change of extension, and a native PipeWire capture helper would produce indexed files at the source.
…Cast portal
Every Linux recording shipped a cursor track pinned to the top-left corner.
`telemetryRecordingSession` samples `screen.getCursorScreenPoint()`, which
returns {0,0} under Wayland — a real recording contained 473 samples, all
`{cx:0, cy:0, visible:true}`. The file looked valid, so the compositor drew its
cursor sprite (WGSL mode 7) in the corner for the whole video. Silent output
corruption.
Wayland exposes the pointer only through the ScreenCast portal's METADATA
cursor mode, where the compositor omits the cursor from the pixels and attaches
SPA_META_Cursor per frame. This adds a helper that negotiates that session and
emits nothing else — no video, no encoding. Video still goes through the
existing path.
`electron/native/pipewire-capture/` is a standalone Rust package, deliberately
outside the crates/ workspace: it is a stdio sidecar like the Swift and C++
helpers, shares nothing with the compositor, and would otherwise pull
ashpd/zbus/png into every addon build. The portal half uses ashpd (pure-Rust
D-Bus, so no libdbus-1-dev); PipeWire is reached through a C shim that dlopens
libpipewire-0.3.so.0, because two thirds of its consumer API is static inline
in headers. The binary links only libc and libgcc — no dev package is needed to
build it and none to run it, which also means it survives a distro shipping a
different PipeWire.
Three bugs were found against a real GNOME session rather than assumed:
- SPA_META_Cursor never arrived. mutter 46.2 declares a FIXED
CURSOR_META_SIZE(384,384) = 589872 bytes; our declared range topped out at
256x256 = 262192. PipeWire intersects the two, the intersection was empty, so
the whole ParamMeta object was silently discarded — stream negotiated, ran,
and reported nothing. The ceiling now matches OBS at 1024x1024, and a test
runs spa_pod_filter over the real POD to keep it that way.
- The stream died after one buffer with "target not found". That string is
WirePlumber's, not PipeWire's: PW_STREAM_FLAG_DONT_RECONNECT sets
node.dont-reconnect, and policy-node.lua destroys the node on that branch
instead of reconnecting. Removed; OBS does not set it either.
- on_process drained to the newest buffer and requeued the rest unread, so
cursor updates arriving on separate zero-chunk buffers were dropped. Every
dequeued buffer is now inspected in arrival order.
Verified on GNOME 46 / mutter 46.2 / PipeWire 1.0.5: metas report
`Cursor:589872`, coordinates track the pointer across the screen, four distinct
cursor shapes were captured with their own hotspots, and sprites are cached by
content hash so repeat samples carry only an id.
There is no telemetry fallback: if the helper is missing or the portal fails,
recording proceeds with no cursor file. No file beats a file full of zeros.
Known limitation, and it is permanent: mouse CLICKS cannot be observed on
Wayland. No portal API reports them and /dev/input is not readable, so
interactionType is always "move" and the click-ripple effect cannot work.
`decode_at` passed `SEEK_SET | 4` to av_seek_frame, with a comment asserting "BACKWARD = 4". AVSEEK_FLAG_BACKWARD is 1; 4 is AVSEEK_FLAG_ANY. This is the only seek in the crate that did not use ffi::AVSEEK_FLAG_BACKWARD — pipeline_windows.rs, pipeline_macos.rs and audio.rs all pass 1. Without BACKWARD, ffmpeg lands on the first indexed position at or after the target rather than the keyframe before it. The forward-scan loop then stops on its own first decoded frame, since `pts >= target` is already true, so it does nothing and decode_at returns the NEXT keyframe. The error is a full GOP. That is why the symptom was asymmetric. The screen track carries a keyframe every ~1.78 s and the webcam every ~6.73 s, so the webcam lands ~4x further out. And live::Player::step catches the webcam up with a monotone forward-only loop, so once it is parked in the future it can never come back — it stops as a still frame until the screen clock reaches it. That is the reported freeze. Third wrong constant found in this file today, after -0x2A2A2A2A standing in for AVERROR_INVALIDDATA and the missing thread_count. All three were hardcoded magic numbers with comments asserting the wrong value; ffi.rs already held the right ones.
`compose_frame` submitted and returned without waiting, then `readback_direct` submitted a second command buffer and blocked in `device.poll(Maintain::Wait)` — which drains the WHOLE queue, so the wait absorbed the entire Kawase blur chain and every layer draw, not just the copy. Meanwhile avcodec_send_frame spent ~8 ms of pure CPU with the GPU idle. The two never overlapped. A ring of staging buffers now keeps frame N's copy in flight while the encoder works on N-1, waiting on the SubmissionIndex that Queue::submit already returns rather than on the whole queue. Measured on AMD Radeon 610M / RADV, 1920x1080, 180 frames, paired A/B in the same binary (depth 1 reproduces the old path byte for byte): light scene 23.98 -> 19.90 ms/frame 41.7 -> 50.3 fps 1.20x heavy scene 24.80 -> 19.62 ms/frame 40.3 -> 51.0 fps 1.26x end to end through the addon, heavy 34.9 -> 42.4 fps 1.22x The poll(Wait) line itself goes from 3.81 ms to 0.008 ms on the light scene and 6.17 ms to 0.007 ms on the heavy one: it has left the critical path entirely. Depth 2, not 3 — depth 3 measured slower (20.48 ms vs 19.62). The render target is deliberately NOT double-buffered: frame N's copy is submitted before frame N+1's compose commands on the same queue, so wgpu's barriers order them without a CPU stall. Correctness is checked by output, not by inspection: the exported MP4s are md5-identical before and after on both scenes, through both the Rust path and the addon, and at ring depth 3 as well — which covers frame ordering, pts contiguity and the flush of the last frame in one check. The three test PPMs are byte-identical too. Two things worth recording for whoever optimises this next. The wall-clock gain is slightly LESS than the wait it removes (heavy: 5.2 ms saved for 6.2 ms of wait); the likely cause is DRAM contention, since a 610M APU has no dedicated VRAM and the overlapped DMA now competes with sws_scale and the encoder for one memory controller — plausible but not instrumented, and it predicts a discrete GPU would do better rather than worse. And the heavy scene GAINS more than the light one (1.26x vs 1.20x), because a heavier GPU frame simply gives the ring more to hide. What remains is now unambiguous: ~12.6 ms of CPU per frame, sws_scale RGBA->YUV420P plus software H.264. macOS avoids the first half by rendering straight into the encoder's buffer.
MediaRecorder writes WebM as a live stream, so the file has no Duration
element — Chromium then reports `duration=Infinity` and `seekable=0..Infinity`
for every Linux recording. `webm-duration.ts` already existed to patch that
header after finalization; this replaces it on Linux with a full remux through
the matroska muxer, which computes the duration from real packet timestamps
instead of trusting a header. Forcing a bogus input Duration of 999999
confirmed it: the output came out 16977.
The remux also writes Cues and SeekHead, which the file lacked. That turns out
to be a nicety rather than a fix — see below.
Same I/O as before (both rewrite the file exactly once) and the existing header
patch stays as the fallback if the remux fails, so a recording can never be
lost to this. Measured: 0.055 s for 7.8 MB / 819 packets, 0.074 s for 44 MB /
18000 packets.
It runs through the compositor addon rather than a spawned ffmpeg:
libavformat.so.62 already ships beside the addon on Linux, and
electron-builder.json5 explicitly refuses to ship a standalone static ffmpeg
("a large binary no runtime code opens", plus LGPL redistribution).
CORRECTION TO 3b61766. That commit added a rewind-and-scan fallback for
`av_seek_frame` failing on index-less recordings, and its message asserted the
seek could not work without Cues. That was wrong. Measured: av_seek_frame
returns 0 for every flag at every timestamp on all four real recordings —
libavformat's matroska demuxer binary-searches by byte position when there is
no index, and Chromium's <video> seeks these files correctly too. An index
takes a seek from 0.229 ms to 0.002 ms on a 10-minute file, which is not
user-visible. The real cause of the original symptom was the seek-flag bug
fixed afterwards in 46b6448 (AVSEEK_FLAG_BACKWARD is 1, the code passed 4).
The fallback in linux_decode.rs is therefore dead code — its `r < 0` branch
cannot fire — and is left in place only as a guard for genuinely unseekable
input.
The gate looked like an oversight now that Linux has real cursor telemetry, and I was about to lift it. Measurement says the gate is right and lifting it would have made the reported "two cursors" bug worse. `cursor: "never"` is not a constraint Chromium implements — it is absent from `getSupportedConstraints()` (verified on Electron 42 / Chromium 148, where the only screen-capture constraints are `displaySurface` and `suppressLocalAudioPlayback`), so WebIDL drops it silently. Below the web API, `DesktopCaptureDevice::Create` wraps every capturer in a `DesktopAndCursorComposer` unconditionally, with no cursor branch; on Linux `prefer_cursor_embedded` is false, so WebRTC asks the portal for METADATA mode and then paints the cursor back in itself. Linux does not even reach the `cursor:` key in useScreenRecorder — that line is inside the win32 branch. Windows and macOS honour the toggle through their NATIVE HELPERS (`captureCursor` to wgc-capture, `hideSystemCursor` to ScreenCaptureKit), not through the constraint, which is a no-op there too. So the gate is precisely "platforms with a native capture helper", and lifting it would ship a control that changes nothing in the pixels while switching the editor's overlay on — producing exactly the two cursors it was meant to fix. `hideSystemCursor` (useScreenRecorder.ts) is inert on Linux for the same reason: it sits inside startNativeMacRecordingIfAvailable, which returns false off darwin. The real fix needs the cursor out of the captured pixels, which needs a native Linux capture helper. Until then the lever that works is `scene.cursor.show`, and defaulting it to false on Linux is a UX decision, not a bug fix — Linux would lose themed, smoothed and click-bounce cursors. Left for a deliberate call.
Stage 2 of the native capture helper starts here: the piece that turns
frames into a file. Nothing calls it yet — the frame source is the next
commit — but the ladder, the muxer and the measurements are in place.
Three backends, tried in order, first one that OPENS wins.
`avcodec_find_encoder_by_name` succeeds for every encoder compiled into
the library including ones this GPU cannot run, so the only honest probe
is to open it with the real parameters.
VAAPI is guarded by a dlsym rather than tried. The vendored ffmpeg 8.1
calls `vaMapBuffer2`, which libva only grew in 2.22; against Ubuntu
24.04's 2.20 the implib trampoline does not return an error, it calls
assert(0) and the process dumps core. A probe that crashes the helper is
worse than no probe.
Vulkan needs RADV_PERFTEST=video_encode before the instance is created,
or ffmpeg reports "Device does not support the VK_KHR_video_encode_queue
extension" on hardware that supports it fine.
Measured here, 1920x1080, Radeon 610M:
convert (BGRA->NV12) 3.62 ms/frame
upload (CPU->GPU) 0.91 ms
encode (h264_vulkan) 0.48 ms
-------
5.01 ms against a 16.7 ms budget at 60 fps
Software (libopenh264) runs the same clip at 40 fps versus Vulkan's 79,
so the fallback still works, just without the headroom.
swscale threading is set to 1 on purpose. 3.80 ms/frame at one thread,
3.84 ms at eight — the request is granted (sws_threads() reads the count
back out) but the conversion is memory-bandwidth bound. An override is
kept because that is one machine at one resolution.
Also fixes the cursor sprite alpha, which Stage 1 left as a documented
guess. It is premultiplied: a captured GNOME sprite had 126
semi-transparent pixels and not one where a colour channel exceeded its
own alpha, which is only true of premultiplied data. PNG wants straight
alpha, so every antialiased edge was being darkened.
One portal session, two outputs: an MP4 with the pointer absent from
every pixel, and the telemetry saying where that pointer was. Chromium's
getDisplayMedia can give neither — it hands back a frame with the cursor
already painted in and no way to know where it went.
Verified on a real 20-second capture at 1920x1080/60: 1213 frames, 0
dropped, 60.05 fps measured, and four crops taken at the positions the
telemetry reported contain no cursor at all.
convert 1.87 ms/frame
upload 1.06 ms
encode 1.00 ms
-------
3.93 ms against 16.7 ms
THE OUTPUT RATE COMES FROM A CLOCK, NOT FROM ARRIVALS. A compositor
sends frames on damage; mutter will go seconds without one while the
user reads a page. One output frame per delivered frame would produce a
file whose playback speed depended on how busy the screen was. So
Capture::advance asks what frame index the wall clock is on and encodes
forward to it, holding the last picture. That is why the encoder splits
conversion from encoding: a held frame costs the upload and the encode
but not the 1.87 ms conversion.
Frames cross the thread boundary through a one-slot mailbox where the
newest wins. A queue would be wrong twice: unbounded at 8 MB per frame,
and every frame it held would add latency to a recording meant to track
the screen. Dropping costs one duplicated frame; queueing costs drift
forever. The count is reported on capture-stopped so a machine that
cannot keep up says so.
Video mode advertises shared-memory buffers only. pw_stream does not map
DmaBuf even with MAP_BUFFERS, so accepting one would leave datas[0].data
NULL and record nothing — not offering it is what makes the compositor
fall back to memfd.
The cursor mode is now a request field rather than METADATA hardcoded,
which is what lets the HUD's cursor toggle mean something on Linux.
Without outputPath the helper is byte-for-byte the Stage 1 cursor-only
session that PipeWireCursorRecordingSession still drives.
…ideo The MP4 now carries AAC tracks. Verified on a live 29-second capture: an 18-second 440 Hz tone played into the default sink shows up as exactly 18 seconds of steady signal in the recorded track, and the two streams end 62 ms apart (video 29.100 s, audio 29.162 s). TWO TRACKS, NOT A MIX. crates/compositor/src/audio.rs already decodes and mixes every audio track it finds on export, and the macOS helper already writes system and microphone separately. Following that avoids resampling two independently-clocked captures onto a common timeline here — each carries its own timestamps and the container reconciles them. AUDIO NEEDS ITS OWN PIPEWIRE CONNECTION. The portal's OpenPipeWireRemote fd is restricted to the one screencast node the user approved, and the ScreenCast interface has no audio option at all. So audio goes through pw_context_connect to the session daemon. PW_KEY_STREAM_CAPTURE_SINK is what makes a capture stream link to a sink's MONITOR ports — what is being played — instead of to a source; without it the "system audio" track records the microphone. The rings are queues, not mailboxes like the video path. A dropped video frame costs one duplicated picture nobody notices; a dropped audio buffer is an audible click. One bug caught by the live test and fixed here: the ring's overflow counter was not reset when the pre-roll is discarded. The audio stream opens before the portal picker is raised, so it records for as long as the user takes to click — past the two-second cap every time. The first run duly reported "78336 system sample(s) were discarded because the encoder could not keep up" for audio that was always going to be thrown away. Both regressions now have tests. An audio stream that fails to open is a warning, not a failure: picture with no system audio is worth more than no recording, and the likeliest cause (a sandbox without the PipeWire socket) is not ours to fix. Vendors 29 more MIT headers, the closure of spa/param/audio/format-utils.h computed the way vendor/README.md prescribes.
Linux recording now goes through the PipeWire helper instead of getDisplayMedia + MediaRecorder, with the browser path kept as an automatic fallback when the helper is missing — the same shape as startNativeWindowsRecordingIfAvailable. ONE PORTAL SESSION, ONE PICKER. SelectSources may be called once per session, so the old arrangement — Chromium's picker for the pixels, the cursor helper's picker for the telemetry — asked the user twice and let them answer differently each time. That was also why the editable cursor never lined up: the telemetry described one stream and the pixels came from another. Now one process produces both, and startCursorRecording refuses to spawn its own helper while a capture is running so the second picker cannot come back. The renderer passes no `source`. On Wayland the portal picker is the source of truth and Electron's desktopCapturer entry is a placeholder, so sending one would be sending a guess. The HUD's cursor toggle works on Linux for the first time. It was excluded because Chromium wraps every capturer in a DesktopAndCursorComposer unconditionally and paints the pointer back in itself — the control would have switched the editor's overlay on without changing the pixels, which is precisely how you get two cursors. The helper owns the video now, so `editable-overlay` asks the portal for METADATA and `system` asks for EMBEDDED. The webcam attach handler was platform-agnostic apart from its guard, so it is extracted and registered for both macOS and Linux rather than copied. Both platforms leave the camera to the renderer's MediaRecorder; only Windows takes it natively. PACKAGING: the helper's ffmpeg libraries go in their own `ffmpeg/` subdirectory, NOT beside the binary. electron/native/bin/linux-x64/ already contains libavcodec.so.62 and friends — the copies whose symbols were renamed to `osff_*` so the compositor addon does not collide with Chromium's ffmpeg inside Electron. With RUNPATH=$ORIGIN the helper found those first and died with "undefined symbol: avcodec_send_frame". The build script's probe caught it; RUNPATH is now $ORIGIN/ffmpeg. The electron-builder filter went from `linux-*/*` to `linux-*/**` for the same reason — one level deep would have shipped a helper that cannot start, and nothing would have noticed until a packaged build. The portal restore token is persisted to userData, so the picker stops appearing on every single recording.
Scrubbing got progressively slower, then froze and killed the app. `decode_at` calls av_frame_move_ref(found, frame) once per frame while decoding forward from the keyframe to the seek target, and never unreferenced `found` first. libavutil/frame.h says exactly what that costs: dst is not unreferenced, but directly overwritten without reading or deallocating its contents. Call av_frame_unref(dst) manually before calling this function to ensure that no memory is leaked. So every intermediate frame in the GOP abandoned its buffers — about 3.1 MB each at 1080p YUV420P, tens of times per scrub. The bug predates this branch, but the new encoder made it spectacular: gop_size was fps * 2, so 120 frames at 60 fps, and a single scrub landing mid-GOP could leak ~370 MB. The GOP is now fps / 2. This file is an editing source and the editor seeks it constantly, so GOP length IS the worst-case scrub cost; the extra keyframes cost a few percent of bitrate on screen content that is mostly static anyway. Also drops the bitrate guess. The renderer was sending computeBitrate(TARGET_WIDTH, TARGET_HEIGHT), and those constants are the app's 4K ceiling rather than the capture size — so a 1080p capture asked for 76.5 Mbit/s and wrote 44 MB for 18 seconds. On Wayland the renderer cannot know the resolution before the portal negotiates it, and the user may well have picked a single window, so it now sends no bitrate at all and the helper derives one from the size it actually got.
Three separate defects, each of which alone reads as "the text renders
strangely".
1. EVERY ANNOTATION WAS ON SCREEN FOR THE WHOLE EXPORT. compose_frame
iterated scene.annotations with no time-window test, so a project with
five captions painted all five, stacked, from the first frame to the
last. macOS and Windows both gate on `t >= start_sec && t < end_sec`.
This is almost certainly what the user actually saw.
2. GLYPHS WERE PLACED AGAINST THE WRONG ORIGIN. cosmic-text's convention
is `glyph.physical((0., run.line_y), 1.0)` with the bitmap's top row at
`baseline - placement.top`. The port passed (0,0), so no baseline was
ever added; added `run.line_top` — a VERTICAL quantity — to the X; and
inverted the sign of `placement.top`. Measured on "Hx" at 40px: the 'H'
ended 14px below the 'x', with no shared baseline. On multi-line text the
second line landed on the same rows as the first, displaced right by
line_top.
3. THE BACKGROUND WAS NEVER DRAWN. Not a blend or format bug — the atlas
is R8 and physically cannot carry colour, and nothing else was emitted.
`spec.background` reached the rasterizer and died in cache_key(). macOS
and Windows bake the plate into the text texture; that is not available
here, so a mode-1 quad (solid colour + rounded-rect SDF, already in
layer.wgsl) is drawn underneath instead.
Also: `align` was parsed and never applied, so centred text — the
editor's default — rendered flush left; and a glyph with a negative
placement.left ('j' in DejaVu Sans) made `dest_x as usize` wrap, smearing
ink onto the right edge of the previous row in release and panicking on
overflow in debug.
Ruled out with evidence, so nobody re-checks them: UV orientation (no
Vulkan Y-flip bug — vs_main maps texel row 0 to uv.y 0), sRGB, and
premultiplied alpha.
The CPU half of rasterize() is split out as build_atlas() so glyph
placement is testable without a GPU — it is pure arithmetic, and it is
exactly where the port went wrong. The two new placement tests fail on
the old code with the measured numbers above.
…d bounce slider WEBCAM STRETCH. compositor_linux.rs drew the camera with `src` wired to [0,0,1,1], mapping the whole texture onto whatever box the layout gave it — a pure non-uniform scale by box_ar / cam_ar. In rectangular PiP the layout already preserves the aspect, so nothing looked wrong; with a circle or square mask, compositeLayout.ts forces width == height, and a 16:9 camera was squashed by 1.78x. `cover_crop_uv` is the shared primitive macOS and Windows already use and it returns the rect unchanged when the aspect already matches, so no correct placement moves. MIRROR. Falls out of the same line: reversing the u interval flips the image. The vertex shader interpolates `src` linearly and fs_main never re-clamps i.uv, so no WGSL change is needed, and after the cover-crop both bounds sit strictly inside the texture where ClampToEdge cannot bleed. `lp.webcam_mirror` had simply never been read on Linux. CLICK BOUNCE. Nothing is broken — the control is inert by construction. Wayland gives an unprivileged process no way to observe mouse buttons (no portal for input events, /dev/input/event* is root:input), so the capture helper stamps every sample "move", CursorTrack::clicks is always empty, bounce() returns 1.0, and frame_geometry multiplies the cursor size by exactly 1.0 for any slider value. The slider is now hidden on Linux, matching how "Shrink on zoom" is dropped a few hundred lines above in the same file rather than shown as a control that does nothing.
layer.wgsl has had the mode-2 drop shadow since the port began — a rounded-rect SDF whose alpha falls off across the spread. What was missing was the Rust half: compose_frame never built a mode-2 LayerCB and never issued the two draws, so `cfg.shadow` and `lp.shadow_scale` were read by nobody and the Shadow slider did nothing at all on Linux. Both shadows use the tuning fractions already shared with macOS/Windows (SCREEN_/WEBCAM_SHADOW_SPREAD_FRAC and friends in frame_geometry), so they are the same relative size on all three platforms at any output resolution. Each is drawn immediately before the layer it belongs to: under that layer, over what is behind it. The camera shadow is skipped in the dual-frame and vertical-stack presets — the camera is tiled flush against the screen there, and a shadow between them would draw a seam. Same condition as macOS. Adds a rendered-frame test that exercises this together with the two previous fixes: shadow on, camera present, and a text annotation with a background. It asserts the annotation's plate is actually on screen — counting pink pixels in the region — which fails outright when the plate is not drawn, rather than only being visible to someone opening the PPM.
Ports the perspective-rotated screen layer, its projected-quad shadow,
and the tilted cursor. Verified independently of the agent that wrote it:
99 lib tests and 8 rendered-frame tests green in this checkout, and the
renders inspected by eye — the iso preset is an unmistakable trapezoid
with the right edge receding, rounded corners following the tilt, and the
full source content present (so crop and zoom really are applied to (s,t)
rather than sampled raw).
None of the maths is new: regions.rs was already cross-platform and its
tilt tests already ran on this host. What was missing was GPU code and
uniform packing.
Two porting traps avoided, both called out in the Metal source:
- mode 8 has NO dst_prev clip test, because dst_prev.xy holds plane_px
in pixels there rather than a 0..1 rect;
- mode 12 reads its spread from mb.y, not fx.x as mode 2 does.
The MSL's two runtime-indexed local-array loops are hand-unrolled rather
than translated literally: naga 24 emits invalid SPIR-V for that pattern
and RADV segfaults inside vkCreateGraphicsPipelines. The HLSL uses a
dynamic array index for sd_convex_quad, which is exactly why the Metal
was the right source to translate from.
The new branches sit BEFORE the mode-2 catch-all `else`, which is
unbounded — anything falling through it silently renders as a drop
shadow.
Mode 13 was optional and got done anyway, because modes 8 and 12 make
tilted zooms reachable and the old cursor path made the pointer vanish
outright under one. That would have been a regression introduced by this
very commit.
Assertions are geometric rather than existence checks: the upright
reference's top edge is flat to 0px while iso/left/right all swing 27px,
and left/right land 1px apart, which is the mirror symmetry the presets
should produce.
Known gap: `regions.rs:956` calls the left/right presets "pure Y
rotations", but they are [-8,-16,-1] and [-8,16,1] — all three axes
nonzero. No preset is truly affine, so quad_inverse_bilinear's
degenerate `abs(k2) < 1e-5 * abs(k1)` branch is ported faithfully but
unexercised by these tests.
… that ignored their own scene I told the user the circle mask was missing on Linux. It is not — I was wrong, and the way I was wrong is worth recording. Two rendered-frame tests I wrote called set_scene() without set_live_params(). compose_frame reads LiveParams, not the raw scene, so webcamShape/padding/effects never arrived: the scene parsed fine and was then ignored, and the renders showed the DEFAULT "rounded" shape. I read those rounded corners in a downscaled frame as "rectangular" and reported a missing feature. This gotcha is documented — it is the first item under "Test gotchas" for this crate — and I walked into it anyway. The new test measures instead of eyeballing. It renders the same scene three times (no camera / rectangle / circle) and diffs each against the no-camera pass, so the diff IS the camera's footprint including its mask, with no need to guess where plan_frame put it or to tell camera pixels from background. Then it checks the bounding box's fill ratio, which separates the two shapes unambiguously: rectangle 256x192 AR 1.333 fill 0.993 circle 256x256 AR 1.000 fill 0.798 (pi/4 = 0.785) So the circle both forces a square box and crops to a real disc. Had this test existed with set_live_params missing, it would have reported identical numbers for both shapes — which is exactly what it did before I fixed it, and is what made the omission obvious.
The two native references genuinely disagree here: Windows centres with DWRITE_PARAGRAPH_ALIGNMENT_CENTER (text_windows.rs:175-176) to match the web overlay's `alignItems: center`, macOS does not, and the Linux port had copied macOS — so text sat at the top of its plate. The web is the product intent, which the user confirmed. The block height is measured from the last laid-out run rather than counting runs times line_height: cosmic-text splits one logical line into several runs when it wraps, so counting runs would overestimate the height of any text that overflows its width and push it up out of centre. Text taller than its box stays anchored at the top instead of overflowing upward, where it would be cropped away entirely.
…hine GGML_NATIVE was ON under a comment claiming it produced a "+avx2 +fma +f16c" baseline. It does not: ggml turns GGML_NATIVE=ON into `-march=native`, so the CPU backend targets whatever CPU ran the build. build-whisper-stt.yml builds the shipped binaries on GitHub-hosted runners, whose x64 images are Xeons with AVX-512 — so a published artifact could carry AVX-512 and die with SIGILL on every AMD before Zen 4 and every Intel consumer part since Alder Lake, and the baseline shifted silently with whatever hardware picked up the job. Same class of bug as the OpenSSL note: an artifact whose contents depend on the machine that built it. Turning GGML_NATIVE off is necessary but not sufficient. ggml derives its instruction-set options from INS_ENB via `option()`, which only seeds a value the first time a build tree is configured — a cache written while NATIVE was ON keeps GGML_AVX2=OFF, and the result is a plain x86-64 binary with no AVX2 at all. Verified: that path produced zero %ymm instructions. So pin the baseline explicitly instead of inheriting it. Ships SSE4.2 + AVX + AVX2 + BMI2 + FMA + F16C, x86-64 hosts only, AVX-512 deliberately excluded. That is the Haswell/Zen 1 floor the installer already requires. OSC_NATIVE_CPU=ON restores -march=native for local benchmarking. Verified on linux-x64: `-msse4.2 -mf16c -mfma -mbmi2 -mavx -mavx2`, 12744 %ymm instructions, 0 %zmm, and the helper still transcribes on Vulkan.
build-whisper-stt.sh stages a symlink farm on Linux and macOS (libggml-vulkan.so -> .so.0 -> .so.0.15.1), and plain `cp` follows every link named on the command line, so each one became a full copy of its payload. libggml-vulkan.so.0.15.1 is 60 MB on its own: measured, the bag was 188 MB instead of 63 MB and the uploaded tarball 59 MB instead of 20 MB. gzip cannot recover any of it — its 32 KiB window is far too small to dedupe copies that sit 60 MB apart. `cp -a` preserves the links; tar keeps them from there (no -h) and untars them intact on the consuming side. Verified end to end on linux-x64 with the real libraries: cp -a -> tar czf -> tar xzf leaves 12 symlinks and the extracted helper still answers GET / and transcribes. `dir/.` rather than `dir/*` so the copy does not depend on the shell's glob skipping dotfiles. -v is kept: the step's log output was the point of it. This shrinks the artifact and the CI download only. The installers are unchanged, because scripts/stage-whisper-stt.sh:65 is a second plain `cp` that re-expands the farm before electron-builder runs. Making that one -a as well would cut ~131 MB from every package — electron-builder recreates symlinks (builder-util/out/fs.js:263) and squashfs stores them natively (verified: a 61 MB farm becomes a 19 MB image), but deb and pacman go through fpm, whose behaviour here I could not verify, so that step wants a real package inspection rather than an assumption.
The user reported the custom cursor only appearing once they opened the cursor settings panel. The cause is broader than the cursor and worth stating plainly: NOTHING pushed the complete param set. Each group was pushed only by the mount effect of the panel that owns it — VideoEffectsPane, LayoutPane, CursorPane — and the inspector renders exactly one panel at a time, defaulting to "effects". So on load the layout and cursor panels had never mounted and their params sat at the addon's compiled-in defaults: cursor size 1.0 for a project storing 3.0, smoothing 0 for 0.67, motion blur 0 for 0.35, webcam shape and mirror likewise. Opening the panel pushed them and it looked like the panel had switched something on. The store's lastParams replay could not save this: it replays keys that were pushed at least once, and these were never pushed at all. pushAllNativeParams() now sends all fourteen keys the addon understands (live.rs set_param_bool/num/str), called from NativeCompositorOverlay — the one component that exists regardless of which panel is showing, and that already owns the view and the scene push. No viewId guard: setNativeParam memoises and setCurrentNativeViewId replays, so mount order does not matter. It cannot fight a slider drag either, because setLive goes through setDocument, so the overlay's document selector already carries the new value on the same tick. The three mount-sync effects are removed rather than left as belt and braces — duplicated pushes of the same value are how the two paths would silently drift apart. UNITS. The helper sends RAW values, matching the old mount effects and NOT the sliders: SliderCell displays cursor.size * 10 and divides by 10 in its handler, with * 100 for smoothing and motion blur. Sending slider-space values here would scale the preview by 10 or 100 on load. The two base-unit constants moved to src/native/paramUnits.ts so the helper and the panels cannot disagree about them. Also fixes the Camera Shape row overflowing its panel. `repeat(4, 1fr)` is `minmax(auto, 1fr)`, so each track's minimum resolves to its content minimum, and "Rounded" is an unbreakable word: four buttons at 64.83px plus three 8px gaps need 283.3px where the panel has 234, so the tracks refused to shrink and the last button was clipped. `minmax(0, 1fr)` plus minWidth:0 on the buttons lets them shrink. The new test asserts NO addon key is missing, transcribing the list from live.rs — a test that checked only the keys we happen to push would have passed against the bug.
Two independent bugs made a Linux recording sound empty. Both are fixed here because either alone still leaves silence. WRONG MICROPHONE. The renderer sent no device at all, so the helper passed no PW_KEY_TARGET_OBJECT and PipeWire linked the stream to the session DEFAULT source. On the machine this was found on that is the headphone jack with nothing plugged into it; the user's built-in microphone was a different node. Measured: the recorded track sat at -61 dB mean while the chosen microphone gave -20 dB. The HUD's level meter moved the whole time because that is getUserMedia, which does honour the picker. This was a documented gap, not an oversight — nativeLinuxRecording.ts said the browser deviceId and a PipeWire node.name are unrelated namespaces and left the field empty. Documenting a hole is not filling it, and I shipped it saying audio worked. There is no id to look up, so the helper now enumerates the graph itself (pw_registry, new in pw_audio.c) and matches the label Chromium reports against node.description — which on a PipeWire system is the same string. Exact node name, then exact description, then containment both ways with the longest description winning so the result cannot depend on registry announcement order. No match means no target, i.e. the old default-source behaviour, plus a warning saying so. Verified live: label "Family 17h/19h HD Audio Controller Digital Microphone" resolves to alsa_input.pci-0000_03_00.6.HiFi__hw_acp6x__source. ONE TRACK, NOT TWO. The MP4 carried system audio and the microphone as separate AAC tracks, following macOS on the grounds that the export mixes every track it finds (audio.rs). The preview does not: it plays an HTML5 <video>, which plays only the default audio track and cannot be told to switch, because Chromium does not implement the audioTracks API. System audio was written first and flagged default, so with nothing playing the preview was silent while the microphone sat in a track nothing would ever select. The Windows helper has always written one mixed track (mf_encoder.cpp has a single audioStreamIndex_ fed by AudioMixer); this now matches it. macOS still has the two-track bug. Mixing consumes min(available) across inputs so neither runs ahead, which taken literally means an unplugged microphone freezes the whole track — its min stays zero forever. AUDIO_STARVE_SAMPLES (a quarter second) breaks that: past it the mix takes the longest input and lets the silent one contribute zeros. Contributions are summed and clamped ONCE at the end; clamping per input would attenuate the mix whenever one source alone is near full scale.
The line that answers "that is not my microphone" was emitted as a `debug` event, which the emitter drops unless OPENSCREEN_PIPEWIRE_DEBUG is set. The app never sets it. So when the user recorded, the resolved node was absent from their log and the fix had to be inferred from a volume measurement instead of read directly. It is now a first-class `audio-source` event carrying the role, what the app asked for, and the PipeWire node actually used — logged unconditionally by the capture session, with a warning when a requested device matched nothing and the session default was recorded instead. Diagnostics for a failure mode that is invisible to the user (audio from the wrong device sounds exactly like a quiet room) do not belong behind a debug flag.
Adding a text annotation created it empty, and an empty annotation renders nothing — so the user added a region and saw no change on the canvas. The inspector's placeholder is CSS ghost text: it never reaches `content`, so it never reached sceneDescription and never reached the compositor either. `annotation.defaultText` added to all thirteen shipped locales (the parity test walks every folder and fails on any key `en` defines alone). The values are proposals, not verified project wording — no "Hello" existed anywhere in the locales to copy a register from. Worth a native speaker's eye, particularly the CJK ones where "こんにちは" may read as more conversational than a UI default should. A default in a script the machine has no font for would render as nothing — the exact bug this is meant to fix, but silent and only for some users. So a compositor test rasterises all thirteen through the real Linux text path and asserts each produces ink. On this machine all pass, Arabic, Japanese, Korean and Chinese included: 677 to 2254 px each. It documents what the running machine lacks rather than pretending to guarantee font coverage everywhere. useTimeline now reads the locale, so its 17 tests need the I18nProvider. One `renderTimeline` helper covers 16; the render-count test keeps its own renderHook because it counts renders itself.
A recording made with the camera on opened in the editor showing the SCREEN video inside the camera's PiP box. Two independent defects, one behind the other. THE CAMERA WAS NEVER LINKED. `cameraTrack.offsetMs` is an integer in the document schema, and both native capture paths derive it from two `performance.now()` readings — a clock with 100 µs resolution, so the subtraction lands on something like -192.80000000447035. `parseDocument` then threw inside the camera auto-link, the catch treated it as a lookup failure, and the asset was saved with `cameraTrack: null`. Four of the last five recordings on this machine lost their camera that way; the fifth measured -243 exactly and kept it, which is what made the bug look intermittent rather than mechanical. Rounded at the recorder so new manifests are clean, and again in `addAsset` so recordings already on disk recover. THE MISSING CAMERA RENDERED AS THE SCREEN. `open_and_seek_clip` opens the screen file as the webcam decoder when the webcam path won't open, so the "webcam" always has frames; `LiveParams::has_webcam` is what says whether they are a camera. It was computed as "webcam path != screen path", which is true for the empty string the app actually sends for an asset with no camera — and the Linux backend never read the flag at all, unlike Windows and macOS. Either bug alone would have drawn the recording into its own corner. Both fixes are covered by tests that fail without them: the rounding in `projectStore.test.ts` and `webcamOffset.test.ts`, the rendered frame in `compose_linux.rs`, which asserts strict equality with the no-webcam preset rather than a threshold. macOS shares the offset defect line for line and is fixed here too.
The Linux compositor only ever drew ONE of the four annotation kinds. A
project could carry arrows, blur masks and images all the way here — the
scene parsed them, `SceneAnnotation` had the fields — and the render loop
skipped anything that wasn't `text`, silently. Worse for the blur: an
annotation whose whole job is to hide something rendered as nothing at
all, so the export showed in the clear exactly what the user marked as
private.
Ported from `compositor_macos::draw_annotations`, mode for mode:
- mode 9 (figure): three round-capped SDF segments, the same trace as
`ArrowSvgs.tsx`. `regions::arrow_local_geometry` was already shared
and tested, so the shader receives endpoints, not a direction to
guess at.
- mode 10 (blur): reads a mipmapped copy of the composed frame. wgpu
has no `generate_mipmaps`, so the pyramid is built by one fullscreen
pass per level with the existing `fs_copy` pipeline. It is the
pyramid that makes the blur — a few offset taps only produce ghosting.
- mode 7 (image): contain-fit, never stretched, cached by annotation id
rather than by path (an image annotation carries a multi-megabyte
data URI, which would be rehashed every frame as a map key).
Annotations now have their own render pass. They must: the blur snapshot
has to be taken after the screen and camera are down but before the first
annotation, and a pass cannot sample the target it draws into. Taking it
once per frame is also what keeps two overlapping blurs from sampling
each other.
`text_anim.rs` was ported, unit-tested and reachable, but nothing on
Linux called it — animated text simply appeared all at once. It now
drives opacity, scale, translation and the typewriter reveal, with the
background plate following the same opacity so a fade doesn't flash a
solid rectangle first.
The four new tests measure on the GPU rather than assert that a draw was
issued: an arrow must cover less than half its box (an unknown mode falls
through to the shadow branch and fills it), a blur must destroy the high
frequencies under it, a mosaic must produce equal neighbours, a 4:1 image
must still measure 4:1, and a fade at 0 ms must be invisible.
Note for anyone writing more of these: `compose_frame`'s third argument
is a FRAME NUMBER, not seconds. Passing 1.0 puts the timeline at 16 ms,
which silently moves every annotation outside its visibility window — the
first version of the animation test passed for that reason and not
because the fade worked. `set_timeline_time` pins it honestly.
EtienneLescot
force-pushed
the
feat/linux-compositor-port
branch
from
July 31, 2026 21:43
513495d to
1085fb0
Compare
None of these were visible before the rebase: each one is a collision
between this branch and something v1.8.0 changed underneath it.
Windows `cargo check` — v1.8.0 made `pipeline_windows::Decoder`
`pub(crate)`, and `tests/compose_linux.rs` imports `pipeline::Decoder`.
Files under `tests/` compile on every platform regardless of what they
test, so a Linux-only test broke the Windows build. Gated with
`#![cfg(target_os = "linux")]`, mirroring the `#![cfg(windows)]` that
`warp_device_cannot_decode.rs` already carries for the same reason.
Lint — one `path.join` in the pipewire helper's build script was split
across lines where biome wants it on one. Formatter only.
Test typecheck — this job fails when the error count grows past a
baseline. v1.8.0 measures exactly 71, the baseline in ci.yml; this branch
measured 81. The ten new ones had three causes, and none of them was a
real type problem in the code under test:
- `nativeCompositorStore.test.ts` (7): the `setCompositorParam` mock was
`vi.fn(() => …)`, so its inferred call signature took no arguments and
`mock.calls[i][1]` indexed an empty tuple. Spelling out the parameters
types the recorded calls.
- `pipeWireCursorRecordingSession.test.ts` (2): `actual.default` on a
node builtin. The default export exists at runtime — it is the CJS
namespace, and the module under test may import it — but `@types/node`
declares only the named exports, so it is read through an index type
rather than dropped from the mock.
- `projectStore.test.ts` (1): `schemaVersion: 6` in an unannotated object
literal widens to `number`, and the document type pins the literal `6`.
`as const` fixes the fixture, and with it the five sibling errors that
were already in the baseline.
That last one takes the count to 66, so BASELINE drops to 66 — the job's
own comment asks for it, and leaving it at 71 would bank room for five
future regressions to slip in unnoticed.
The `defaultOf` helper this started as is worth a warning: `vi.mock` calls
are hoisted above every top-level statement, so a module-scope helper is
still in its temporal dead zone when the factory runs and the mock dies
with "Cannot access 'defaultOf' before initialization". A `type` alias is
erased and safe; a `const` is not.
1220 renderer tests pass, 112 + 17 + 3 Rust, lint clean, tsc clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's in this branch (12 commits on top of
#162's tip)What's in each commit
Spec —
linux-compositor-port.md(614 lines): target architecture, three-axis decision matrix (render/decode/encode), the AVFrame carrier contract generalising the four-field seam #162 introduced, the WGSL shader port inventory (1:1 with HLSL's 9 entry points), thecosmic-textdecision fortext.rs, build/CI plumbing for Linux, verification methodology (3/255 cross-backend tolerance), WP0–WP7 milestone plan.WP0 — Cargo gating.
wgpu = "24"pinned to thewindows = "0.58"-compatible line. D3D11 modules gated#[cfg(windows)]incompositor/src/lib.rs.compositor-view-napisplit: a small platform-agnostic stub plus a#[cfg(windows)]-gatedwindows.rs.build.rscross-build detection viacfg!(host_os)vsCARGO_CFG_TARGET_OS; on cross-build, skipscc/bindgenand writes a cachedffi.rs(CI on Linux regenerates the real bindings).WP1 —
vk.rs.VkGpu { device, queue, backend, adapter_info },create/create_blocking/create_auto,probe()cached viaOnceLock,classify()detecting lavapipe by adapter name on Linux and WARP on Windows — both routed toBackendKind::CpusoprobeBackend()keeps its three-state semantics for free.WP2 + WP3 —
vk_frames.rsand the slice renderer.VkFrameTex { y: R8Unorm, uv: Rg8Unorm, width, height }.VkFrames::present(src)does the swscale → upload of NV12 split into two wgpu textures.vk_decode.rs::SwDecoderis a software decoder.vk_render.rs::render_slice_c1runs the full pipeline:vk_frames::present→ WGSLlayer.wgsl(modes 0/1/2, BT.709 limited YUV→RGB, rounded-corner SDF, smoothstep feather) → copy_texture_to_buffer + map_async + pollster::block_on → RGBA8 → PPM.vk_cross_goldenintegration test: decodes frame 180 of the fixture, renders, writes PPM, logs stats. The slice produces a real, recognisable GitHub README frame on this Windows machine — mean RGB ≈ 232 over 6.22 M channels, dimensions 1920×1080.WP4 arch —
CompBackendtrait. Trait defined inbackend/mod.rswith 10 methods covering thecompose_framepath.LayerCBextracted tobackend/mod.rsas the platform-neutral constant buffer (matches HLSLcbuffer Layerand WGSLstruct Layerbyte-for-byte).D3d11Backend(Windows) wraps the existingCompositorand forwards — zero behavioural change on Windows, all 80 unit + 3 integration tests still pass.WgpuBackend(Linux) implements the trait, withvk_frames/vk_renderprimitives for the rendering.WP4 effects — partial.
WgpuBackendimplementsdraw_solid(mode 1),draw_shadow(mode 2, SDF-based),draw_quad_shadow(mode 12 wrapper for the AABB case), andblur_bg(the full Kawase chain — 3 down passes + 3 up passes, 2.2 spread, all 6 render passes share aRenderCtxwith 3 pyramid textures and 2 dedicated pipelines). The shader isvk_shaders/blur.wgsl(5-tap average down + 5-tap weighted up — matches HLSLps_kawase_down/ps_kawase_up). The remaining HLSL modes (image_bg / cursor / motion blur / tilt / annotations) are marked as WP4+ follow-ups in commit messages.WP5 —
text_cosmic.rs. Linux text rasterizer usingcosmic-text0.19 (pure Rust: rustybuzz shaping, swash rasterization, fontdb font discovery). SameTextSpecshape and same FNV-1acache_key()algorithm astext.rs(Windows DirectWrite).TextRasterizer::new(device, queue)+rasterize(&mut self, &TextSpec) -> Result<RasterizedGlyphs>builds aBuffer, shapes it, iterateslayout_runs()to pull each glyph image fromSwashCache::get_image(...), and uploads the per-glyph alpha into aR8Unormwgpu texture. Text is consumed byWgpuBackend::draw_annotations(still unimplemented, lands with thecompose_frameport).WP6 —
pipeline_lin.rs. Linux pipeline arm.Decoder::open(path, &VkGpu)opens the fixture, preparesvk_framesfor upload.next(frame_idx)decodes + uploads.VideoEncoder::open(codec, w, h, fps, bit_rate)opensh264_vaapi(NV12) when/dev/dri/renderD128is present, elselibopenh264(YUV420P) — the Linux counterpart of the Windows vendor stack.VideoEncoder::send_composited(carrier)reads the carrier's NV12 split, copies to NV12 contigu (for VAAPI) or swscales to YUV420P (for libopenh264), thenavcodec_send_frame.run_compositedis a skeleton — the full MP4 muxer (avformat_new_stream + write_header + write_trailer) needs the Linux shim forAVFormatContext.streamsaccess, deferred to a follow-up.WP6 — module restructuring.
pipeline.rs→pipeline_win.rs,live.rs→live_win.rs, with cfg gates.pipeline_lin.rsandlive_lin.rsare the Linux counterparts.poc-d3dandcompositor-view-napi/src/windows.rsupdated to use the renamed paths.WP7 partial — Linux napi arm.
crates/compositor-view-napi/src/linux.rsexposes the same#[napi]surface as the Windows arm (create_view / read_frame / destroy_view / set_scene / set_live_params / dump_ppm / probe_backend). It uses the slice renderer (one frame per call) as the production path —vk_cross_goldenproves this works end-to-end.probeBackend()returns"none"until the fullLiveViewlands; the slice path is real but not exposed as a live preview stream yet. The compositor-view-napiwindowsdependency is nowcfg(windows)-gated so the Linux build doesn't pull it.Decisions baked in (mirror the spec §4)
poc-nativePOC rancomposite.wgslunchanged on Vulkan with pixel-identical output. The 2026-07 wgpu rejection indecisions.mdwas a Windows-encoder problem, not a Linux-render problem.cosmic-textfortext.rs— pure Rust, 13-locale shaping parity, no system C deps.Backendenum stays semantic —probeBackend()is platform-agnostic on the Electron side; no UI work needed.CompBackendtrait covers thecompose_framepath. Out-of-trait methods (encode-sidefs_pass, readback, blur_bg of secondary effect sets) stay onCompositordirectly until migrated.Verified
cargo check --workspace(Windows host) — green. 86 unit + integration tests pass — the D3D11 path is unchanged.cargo check --workspace --target x86_64-unknown-linux-gnu— green. WGPU + the Linux pipeline arm + the Linux napi addon + all module splits compile on Linux (usingOPENSCREEN_BINDGEN_CACHE_PATHfor the cross-build; CI on Linux regenerates the real bindings).What's deliberately still in follow-up commits (within this same PR if review permits)
The branch ships a real Linux compositor path: decode → upload → render → readback, all proven on real hardware via the slice. The remaining work is filling in the production
compose_frameorchestration math on Linux (which today still goes throughcompositor.rs::compose_frame, a Windows-only path) and the corresponding readback + export side:compose_frameorchestration port — 800 lines of pure Rust incompositor.rs(placements, zoom state, cursor follow, motion-blur supersampling). The dispatcher toWgpuBackenddraws is the next commit; once it's in, the full C1..C8 effect set works on Linux end-to-end.live_lin.rsLiveView — the render thread that produces frames continuously, not one-shot. The skeleton exists; wiring the napi to it (withSend/Syncfor the GPU handle and a generation counter) is the next commit.pipeline_lin::run_compositedis a skeleton today. The full muxer (avformat_new_stream, write_header, write_trailer) needs the Linux shim forAVFormatContext.streams— same pattern as the Windows shim, just with the Linux AVFormat bindings.WgpuBackend::draw_layerinfrastructure. They follow in subsequent commits; the trait architecture is in place to absorb them.build-linux-compositor-addon.mjs,fetch-ffmpeg.mjsLinux arm, electron-builder extras for Linux. These are CI plumbing, not blocking the core port.Each follow-up commit is small (100-300 LoC) and reviewable; they keep landing in this same PR per the user's "all WP should be on this PR" directive.
Cross-backend parity verification path
vk_cross_goldenslice test produces a valid frame on this Windows AMD iGPU (mean RGB ≈ 232 over 6.22 M channels, dimensions 1920×1080, written tocrates/compositor/target/vk-slice-c1.ppm). That's the proof the WGSL+vk+wgpu+swscale+YUV→RGB+SDF+feather chain works on real hardware.poc-d3d --release --cfg C1 --backend hardware --frames 1, deferred to the bench-side follow-up). The slice produces correct output in isolation; the diff infrastructure (OPENSCREEN_VK_GOLDEN,diff_stats, the 3/255 gate) is in place and CI runs it once both PPMs are committed.Branch / PR state
Related
Backendseam,cpu_frames.rs,probeBackend(), the iso-render tolerance baseline this port is measured against.technical-documentation/architecture/native-compositor.md— the engine today; "Linux port" subsection now links the spec.technical-documentation/architecture/linux-compositor-port.md— the spec.technical-documentation/engineering/rendering-performance.md— C1..C8 measurement methodology, reused verbatim for the Linux bench numbers when the bench lands.